[SPARK-59398][ML][SQL] Add a SQL expression for ML vector scaling and shifting - #58692

What changes were proposed in this pull request?

This PR adds an internal VectorScaleShift Catalyst expression, registered as
ml_vector_scale_shift, that computes vector(i) * scale(i) + shift(i) from the SQL struct
representation of an MLlib vector and non-nullable-element array<double> scale and shift inputs.

The expression supports interpreted and code-generated evaluation for dense and sparse vectors
through a shared MLExpressionUtils.scaleShift implementation. Code-generated evaluation
caches non-null literal scale and shift arrays as primitive double[] references.
A null scale column value is treated as an identity scale, and a null shift column value is treated
as a zero shift. When both are null, the input vector is returned unchanged. It preserves sparse
output only when the input vector is sparse and the shift is null; any non-null shift produces a
dense vector. It also adds private ML helper functions for column and constant-array scale/shift
inputs. This PR does not use the expression in ML implementations.

Why are the changes needed?

Several ML feature transformations perform per-feature scaling and shifting through Scala UDFs.
This expression provides a reusable Catalyst building block for future optimizations without
deserializing vectors into MLlib objects.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

New tests cover interpreted and code-generated evaluation with dense and sparse vectors, null
scale and shift handling, non-null array element enforcement, sparse output preservation, nonzero
shifts, null and empty vectors, infinite and NaN values, and mismatched dimensions.

build/sbt 'catalyst/testOnly *VectorScaleShiftSuite'
build/sbt 'mllib/testOnly *FunctionsSuite -- -z "vector_scale_shift"'
./dev/lint-scala

Was this patch authored or co-authored using generative AI tooling?

Generated-by: OpenAI Codex (GPT-5)